Measure Platform User Guide
Measurement and Data Analysis
Platform
••••••••••••••••••••••••••••••••••••••••••••••••••••••
Measure Platform Overview
The Measure platform is a tool dedicated to measure, analyse, and visualise the metrics to extract and show information of the software engineering processes.
· Implement the tools for automatically measure software engineering processes during the whole software lifecycle by executing measures defined in SMM standard and extracted from a catalogue of formal and platform-independent measurements.
· Provide methodologies and tools which allow measure tools provider to develop a catalogue of formal and platform-independent measure.
· Implement storage solution dedicated to measurements resulting of measure execution in big data context.
· Implement visualization tools to expose the extracted results in an easy-readable fashion, so allowing a quick understanding of the situation and the possible actions that can be taken to improve the diverse stages of the software lifecycle.
· Implement an extension mechanism dedicated to the integration of external analysis tools will provide long terms analysis and predictive evaluations on collected measures.
· Implement of an Extended API allowing to facilitate the integration on Measure Platform with external tools and services.
The Measure platform provides services to host, configure and collect measures, to store measurements and analyse them. These measures are first defined in SMM standard using the Modelio modelling tool and its extension dedicated to SMM modelling. They are packaged under an executable format as Measure Definition (For more details related to measure execution format, please refer to the section 2.4.1 of this document).
Measures are registered and stored on Measure platform in order to initiate the collect of measurements, the next step consists on defining instance of measure based on measure definitions. A measure represents a generic data collection algorithm that has to be instantiated and configured to be applied on a specific context. For example, a measure which collects data related to an SVN repository must be configured by the URL of this repository. Next the Measure Platform can start collecting measurement (data resulting of the execution of an instantiated measure). Collected measurements are stored on a No SQL designed to be able to process a very large amount of data. To collect measurements, the direct measures can delegate the collect work to existing Measure tool.
Stored measurements are presented directly to the end user following a business structured way by the Decision-making platform, a web application which allows organising measures based on projects / software development phases and display its under various forms of charts. The measurements can also be processed by analysis tools to present consolidated results.
The collect of measurement goes through two kinds of connectors that can be deployed on the Measure Platform: The Measure and the Monitoring Applications.
· A Measure is a small connector allow to collect measurements. A Measure can be Direct (Collect of measurement in physical world), a Proxy (Ensure communication between a Measurement Tool and the Platform) or Derived Measure (Measure calculated by the aggregation of existing Measures).
A large set of Measure able to collect measurement form Market Tool or form specific measurements tools are provided with this Platform. This Measures can be found at this URL:
https://github.com/ITEA3-Measure/Measures/releases
·
A Monitoring Application is
composed of a set of measures which address the same data source (Ex: a GitHub
Repository). These measures are packaged together and, deployed a project,
provide automatically a complete Visual Dashboard related to this Data Source.
The Measure Platform support two kinds of extensions which complete the functionalities provided by the Core Platform
· Measurement Tools: External tools that collect or calculate measurements from a specific data source. Measurements Tools are delivered with set of associated Metrics and Monitoring Applications which make the link between the Platform and the Measurement Tool.
· Analysis Tools: Set of external services which work on the historical measures values in order to provide advanced and valuable analysis function to the platform. In order to support a large set of analyses services and do not limit to it a specific technology, the Analysis Tools are external processes. The analysis tool is integrated to the platform using a specific API. This integration includes embedded visualisation provided by the analysis service into the platform
The Analysis and Measurements Tools are packaged separately from the platform and are
provided with specific installation guides and user guides.
Project Catalogue
|
The Project Catalogue list all Monitoring projects currently accessible by the connected user. · The creation of a new project is performed in this view. · Open an existing project. · A metric overview can be associate to the Project. |
|
|
The New Project button allow to create a Project. · Name: Name of the project · Description: Description of the Project · Creation Date: Creation Date of the Project · Project Image: URL of an image which will be associate to the project |
|
|
By clicking on the View Edition button of each project card, it possible to associate an overview to the project. This overview can be: · A metric collected by the project. · A customised content. |
|
Data Source
The Data Sources is the is the channel through which the platform collects data
of the project. Data Sources are
connector between specific data providers like external application or remote
services and the Platform. There two kind of data sources: The Measurements
Application and the Measures.
·
The Measurement Application is
the simplest way to collect, visualise and analyse data in Measure Platform. An
application addresses a specific functional domain or Tool and allow to
automatically collect data from this source and provide a pre-configured
dashboard which allow to consult this data.
· A Measure is a small connector allow to collect measurements. A Measure can be Direct (Collect of measurement in physical world), a Proxy (Ensure communication between a Measurement Tool and the Platform) or Derived Measure (Measure calculated by the aggregation of existing Measures).
Application Data Source
|
As for Individual Measure, a Measurement Application can be deployed and configured in the project. Once deployed, the measures associated with this application will be automatically configured and collected and a specific dashboard will be integrated in main project view to visualise the collected information. · The Register Application Button allow to create a new Measure · Existing Application can be updated or deleted from this view. |
|
|
To deploy a new application, in project, it’s required to fulfil measures parameters (Ex: A GitHubRepository Application required the URL of the GitHub repository which will be Monitored) A – Select the kind of application to deploy B – Name of the measure C – Fulfil parameters required by the measure to manage connection with the data source |
|
Once configured, a measure cans be executed in many ways:
Activate: This button allows you to activate the application. Once activated, the application will be collected periodically by the platform and one or several pre-configured dashboards will become avail be on the dashboard view
Measure Data Source
|
To collect measurements, a measure of the Catalogue must be added and configured in the project. The Measure Configuration view list all measure currently deployed on the project · The Register Measure Button allow to create a new Measure · Existing Measures can be updated or deleted from this view. |
|
|
To deploy a new measure, in project, it’s required to fulfil measures parameters (Ex: A Git Hub measure required the URL of the GitHub repository which will be Monitored) A – Select the kind of measure to deploy B – Name of the measure C – Description of the measures D – Fulfil parameters required by the measure to manage connection with the data source E – Configure measure scheduling: Measures will be executed periodically by the platform to collect measurement. This Feld allow you to collect periodically the frequencies of this collect. |
|
Once configured, a measure cans be executed in many ways:
A – Activate Scheduling: If scheduling period has been configured, this button allows you to activate the scheduling. The measures will be collected periodically by the platform.
B– Create Default Measure View : Measures come with pre-configured visualisation. This Button allow to create automatically a view for this measure on main project dashboard
C – Test the measure: the measure will be executed once for testing purpose. The measure execution result will be displayed but not stored in the measurement database.
D – Execute the measure once. The measure execution result will be display and stored in the measurement database.
E – Edit the measure configuration.
F – Delete the measure.
Project Configuration
|
The Project Configuration page allows to configure the list of metrics, measurement application and Analysis Tools used in the project. A – Edit project properties B – Delete project C – Register analysis pools D - Configure registered analysis tools |
|
|
Analysis Tools are external analysis services integrated into the Measure Platform. To use one of these services in a project: · Click on Activation button to activate an analysis service on this project. Once activate, the analysis tool can be configured using the new dedicated view. Please refer to the documentation of this tools for more details about configuration of a specific analysis service. |
|
Dashboards
The Measure Platform provide several Dashboard in which collected measurements can be visualised. This dashboard is dynamic and can be configured by the project owner to visualises data sources.
The platform provided different kind of dashboards:
· Default Project Dashboard: Each project provides at least one dashboard which allow to visualise collected measurements. This dashboard is shared by all project’s members and configurable by the project owner.
· Application Dashboard: When a measurement application is deployed, one or several non-editable dashboards are automatedly provided by this application.
· Personal Dashboard: Each project user can create and configure dashboards which will stay strictly personal an allow him to have a specialised view of data collected in the project.
· Shared Dashboard: Projects Owner have the ability to configures Dashboards and share it with others project members.
· Kibana Dashboards : Dashboards directly imported form Kibana.
The Main Project Dashboard allows to visual collects measurements. It can also display synthetics information from analysis services or customized contents. The Main Dashboard is shares between all project users, but cans be only configured by the Project Owner.
A – Existing visualisation of measurement
B – Activate / Disactivate the edition model of the dashboard
C – Add a new visualisation in the dashboard
D – Update / Delete existing visualisation
Platform Analysis tool has the possibility to provide some simplified visualisation of the result of their analysis. This view is called Analysis Card. To integrate an Analysis Card, switch the dashboard in edition model and click on the Add Analysis Card button.
|
To integrate an Analysis Card: · Select the Analysis Card form the list of cards provided by analysis tools deploy on the current project |
|
Create various kinds of graphics like bar
chart, line charts and area chart for collected datas. This service allows to
visualise simple measurements that are numerical
measures periodically collocated and visualise them as Bar Charts, Line Chart,
Area Chart or Single Values. To create more complex visualisations, please
refer to the next section “Integrate a Kibana Visualisation”.
|
To visualise the collected measurements as a chart: · Measure: Select the Measure to visualise. The measure configure on this project will be listed yea. · Value to Display: If the data model of the measure contained several numerical values, you can select the value you went to visualise. · Date Index: Select the index used as x axis of the graph: If the data model of the measure contained several data, select the date used as x axis. ·
Chart Type: Select the
type of char used in visualisation (Bar Charts,
Line Chart, Area Chart or Single Value) ·
Display Date From: period of time covered by the visualisation
(ex: 1 year) ·
Aggregate Data By: Select how data points will be aggregated (ex:
1 dot per week) ·
Graphic Size: Initial size of the visualisation. ·
Color: Main color of the graph. · Auto Refresh: Updated automictically the displayed value over time |
|
|
To visualise the collected measurements as a simple data: · Measure: Select the Measure to visualise. The measure configure on this project will be listed yea. ·
Value to Display: If
the data model of the measure contained several numerical values, you can select
·
Graphic Size: Initial size of the visualisation. ·
Color: Main color of the graph · Auto Refresh: Updated automictically the displayed value over time |
|
|
To visualise the collected measurements as a data table: · Measure: Select the Measure to visualise. The measure configure on this project will be listed yea. · Value to Displays: If the data model of the measure contained several numerical values, you can select the value you went to visualise. ·
Graphic Size: Initial size of the visualisation. ·
Color: Main color of the graph. ·
Table Size : number of lines of the table. · Auto Refresh: Updated automictically the displayed value over time |
|
The Measure Platform used the Kibana tool to manage visualisation of collected measurements. Kibana provides a complete editor which allow to create complex visualisations from a set of measurements. If you plan to visualise complex data, you will have to create a Kibana visualisation that will be integrated into the dashboard.
To create a Kibana visualisation, switch the dashboard in edition model and click on the Add Kibana View button.
|
The Create new Visualisation button will open the Kibana editors. This editor will allow you to create complex visualisation on your data. For more information related to Kibana, please refer to the documentation of the tool: https://www.elastic.co/guide/en/kibana/current/index.html To integrate a Kibana Visualisation: · Kibana Visualisation: Select the Kibana visualisation to integrate: The list of existing saved Kibana visualisation will be available. · Graphic Size: Select the size of the visualisation ·
Display Date From: period of time covered by the visualisation
(ex: 1 year) |
|
|
The current configurable dashboard allows you to integrate customised content like images, descriptions or others test using the HTML 5 format. To integrate a customised content, switch the dashboard in edition model and click on the Add Personalised Content button. · View Data: integrate your HTML code which can contains texts, images, media … |
|
Additional Dashboards
On additional dashboard, it is also possible to integrate directly a complete dashboard created with Kibana in the same way as for the measurement visualizations.
|
|
Notification Service
The Measure Platform includes a notification service which allows to notify the user of critical event related to the project. Notification notifications are of various types and sent by the platform or the analysis tools.
Measure and Application Catalogue
|
The Measure Catalogue contains the list of Measures supported by the Platform. Organised in a category, each measure is characterised by is name, a description, the scope (configuration parameters) and the data model of measurements return when the measure is executed. The catalogue allows to: · Upload a measure in the platform form the packaged measure as zip file. · Remove a measure of the platform. In this case, all instances of this measures used in projects are deleted to.
|
|
As for Measure Catalogue, the Application Catalogue contains the list of Measurement application supported by the Platform.
The Application Catalogue allows to:
· Upload a measurement application in the platform form the packaged measure as zip file.
· Remove a measurement application of the platform. In this case, the Application will be removed from all projects which used the Application.
Data Analysis
The analysis of measurements collected by the Measure Platform are delegated to Analysis Tool deeply integrated to the platform. The Analysis Tool are set of external services which work on the historical measures values in order to provide advanced and valuable analysis function to the platform. In order to support a large set of analyses services and do not limit to it a specific technology, the Analysis Tools are external processes. The analysis tool is integrated to the platform using a specific API. This integration includes embedded visualisation provided by the analysis service into the platform. The Table below contained a not exhaustive list of Analysis tool currently integrated to the Measure Platform.
|
Quality
Guard Notification Service |
Quality Guard allow to define quality constraints which allow to compare in real-time measures collected by the platform to predefined measure thresholds and send notification if these thresholds are exceeded. |
|
STRACKER Prediction and Forecasting |
Metrics
prediction, metrics correlation, and metrics forecasting services integrated
to the Measure Platform |
|
MINT Recommendation Service |
The Metrics
Intelligence Tool (MINT) is a software solution designed to correlate metrics
from different software development life cycle in order to provide valuable
recommendations to different stakeholders impacting the software development
process. |
|
|
|
|
Metrics
Suggester Measurement Plan Automation |
The Metrics
Suggester provides a framework to automatize the suggestion of
software metrics based on an
initial measurement plan. To do
so, the framework needs an initial configuration from the user
to determine the metrics
range to be analysed and the classifier. |
|
M·ELKI Clustering service |
M·ELKI is a set of web services that makes possible to select, configure, process and visualize results of several clustering algorithms provided from the ELKI Java library. |
The Analysis and Measurements Tools are packaged separately from the platform and are
provided with specific installation guides and user guides.
In order to integrate deeply the analysis tools into the Measure Platform, the analysis tools provide web pages which will be embedded to the platform. Each of these views are defined on the platform side by a specific URL. For project specific views, this URL is different for each project. You will see below the list of view which can be provided by the analysis tool and embedded into the Measure Platform.
·
Global Configuration Page (optional): If
the analysis tool requires a way to provide some configuration interface which
will be shared by all project, it can provide a global configuration web page.
· Project Specific Analysis Configuration page: Configuration page which are specific for each project. This page is embedded into project configuration page and allow to configure the analysis service provided by the external analysis tool.
Figure 1 : Analysis tool configuration page
of Quality Guard Analysis tool
·
Analysis Tool Main View: Main view of
the analysis tool which are specific for each project. In this view, the
analysis service.
Figure 2 : Main view of the Quality Guard
Analysis Tool
· Dashboard Card: Optional small view which can be integrated to projects dashboards in order to provide some key information to project managers related to the service provided by the analysis tool.
Users Management & Access Right
|
To use the Measure Platform, you are invited to create a User Account. In front page of the platform, clink on the Register button. The registration page will invite you to provide you Name, Email, and password. An email send at the provided address will allow you to complete the registration process |
|
By default, a platform user has access to projects he has created and configured. It’s possible to provide access on specific project to other users registered in Measure Platform. We distinguish two different Roles on a project the Project Owners and Invited Users with specific rights. The user which create the project is the Project Owner of this project.
|
Project Owner Rights |
Invited User Rights |
|
·
Show the project ·
Configure Data Sources ( Application / Measure) ·
Configure Analysis Services ·
Use Analysis services ·
Configure the default project dashboards. ·
Share Dashboards with others projects members ·
Create in own dashboards with specific visualisations ·
Invite another user on the project ·
Design invited user as Project Owner. |
· Show the project · View the default dashboard created by the project owner · Create in own dashboards with specific visualisations · Use Analysis services |
Platform Administration
The Platform administrator (registered user with administrator role) has access to serval administration services to manage the server. These services are accessible via the Administration menu.
|
User Management: Service allowing to create new users, delete existing users and manage user roles. |
|
|
Health Check: Service allowing to check the availability of resources consumed by the server (Disk Space, Memory, etc.) |
|
||
|
|
||
|
|
|
||
|
|
||
|
|
Extend the Platform
The Measure platform can be extended following three ways in order to support new data sources, new way to visualise collected data or now data analysis services.
A Measure is a small and autonomous java program based on the SMM specification which allow to collect measurements. The Measures make the link between a Measurement Tool, a Remote Service, a Captor or any others kind of data sources and the Measure Platform.
If you plan to monitor measures that are not currently supported by the measure platform, you will probably have to develop your own Measures.
There are two kind of Measures:
· The Direct Measure (Collect of measurement in physical world), a Proxy (Ensure communication between a Measurement Tool and the Platform) or
· The Derived Measure (Measure calculated by the aggregation of existing Measures).
For more information related to Measure development, please refer to the Measure Platform Developers Guide.
An Application is a set of Measures aggregated together in order to address a functional requirement. The application is associate with a visual dashboard which directly integrated into the Decision-Making platform when the Application is deployed on a project.
For more information related to Measurement Application development, please refer to the Measure Platform Developers Guide.
In order to support a large set of analyses services and do not limit to it a specific technology, the Analysis Tools are external processes. Although external, we wanted a deep integration between the platform and the analysis tools. We solved this issue in the following way:
· The Measure platform provides a REST API which allows an analysis tool to register it on the platform, to receive notifications from the platform and access to information related to project defined and measure collected by the platform.
· On its side, the analysis tool provides some web pages which will be embedded into the platform web application.
For more information related to Analysis Service development, please refer to the Measure Platform Developers Guide.